Color superposition and mixing of light beams for a video display system

ABSTRACT

Low-cost complex plastic optics allow biocular viewing of video images generated by a single electro-optic display device, such as in a head-mounted display (HMD) for commercial or medical viewing applications. A dual off-axis configuration uses nearly collimated illumination optics and intermediate imaging optics to fill both eyepieces from a single display device without the need for a beamsplitter. Multiple illumination schemes are provided for either monochrome or color, and in either two-dimensional or time-sequential true stereographic presentation. Light from multicolor sources is superimposed, mixed, and homogenized by mixing light cones with diffractive collectors. Offsetting color overcorrection and undercorrection of individual optical elements achieves overall chromatic correction with minimal optical element complexity; A wireless video signal interface eliminates excess cabling. Additional features include lightweight achromatic eyepiece construction and interchangeable lenslets for peripheral vision correction. Multiple optical elements are injection-molded as a unitary plastic structure, thereby reducing cost and complexity.

This application is a divisional application of U.S. patent applicationSer. No. 09/241,828, entitled "Color Superposition, Mixing AndCorrection For A Video Display System" filed Feb. 1, 1999, which is adivisional application of U.S. patent application Ser. No. 09/056,934,entitled "Biocular Viewing System With Intermediate Image Planes For AnElectronic Display Device" filed Apr. 6, 1998, now U.S. Pat. No.5,926,318.

FIELD OF THE INVENTION

The invention relates generally to image display systems and moreparticularly to dual off-axis stereographic systems with multipleillumination sources for biocular viewing of single microdisplays.

BACKGROUND

High quality, convenient, cost-effective medical remote imaging hasgained increasing importance during recent years. This is particularlytrue of imaging during surgical procedures, most importantly minimallyinvasive procedures in which direct viewing of the surgical field by thesurgeon is difficult. For example, a method for performing coronaryartery bypass relies on viewing the cardiac region through athoracoscope or other viewing scope (see for example Sterman et al. U.S.Pat. No. 5,452,733 and Gifford, III et al. U.S. Pat. No. 5,695,504). Byway of further example, a surgeon may need to perform a delicatevascular- or neuro-microsurgical reconstruction through a minimalincision with the aid of remote viewing. Minimally invasive surgicalprocedures and their related need for remote imaging are now common inorthopedics, ophthalmology, urology, gynecology, anesthesiology, andother medical disciplines.

In a conventional surgical environment, remote imaging is accomplishedby attaching a video camera to an endoscope, laparoscope, or otherminimally invasive instrument and transmitting the video image via cableto a conventional CRT video monitor. This is typically cumbersome in acrowded, brightly lighted operating room, where surgical team membersare frequently milling around and the surgeon's view of the image screenis often obstructed. Additionally, the CRT monitor is incapable ofproviding the surgeon with critical depth perception, since it is notstereographic.

Head-mounted displays (HMDs) potentially offer a method for convenientmedical remote viewing without obstruction of the image by the cluttertypical of the operating room. While head-mounted displays have beendesigned, developed and deployed in military applications for manyyears, such displays are generally bulky, expensive,application-specific devices that are not well suited to commercial orsurgical applications.

With the advent of inexpensive and increasingly complex commercialcomputing power and computer graphic devices and software, there hasbeen increasing interest and activity in the field of commercial HMDdevices. A number of such devices are presently available, but becauseof the high cost of appropriate display components and the generallycumbersome mechanical nature of the headgear, these devices aregenerally low in resolution and unattractive for professional computingapplications.

High-resolution display device components are now emerging, that cansignificantly enhance commercial HMDs and related applications. However,they require integration into an ergonomic, well engineered andeconomical design. In the case of professional and consumer computingapplications, visual quality and comfort are critical to long-termacceptance. A computing environment generally includes the use of akeyboard as well as peripheral devices and supporting paperwork.Therefore peripheral vision is also an important consideration.

A compact HMD system requires a very small display device, such as thosefound in modern camcorder viewfinders, but with significantly higherresolution. A number of such devices are now becoming available,including transmissive and reflective liquid-crystal microdisplaydevices and micro-mirror devices having resolutions at or in excess ofVGA quality (640 pixels by 480 pixels) with pixel sizes on the order of15 microns or less. Most of these devices exhibit satisfactoryimage-contrast only when illuminated and viewed at narrow angles ofincidence, which compromises field of view, eye relief, and viewingcomfort.

Due to the base costs of their materials, such devices are expensive forcommercial applications, even in high volumes. In particular, forstereographic or other binocular applications, the use of dual displaydevices for two eye channels results in a high cost. A medicalstereographic HMD system having dual display devices is described inHeacock et al. "Viewing Ocular Tissues with A Stereoscopic EndoscopeCoupled to a Head Mounted Display (HMD),"http://www.hitl.washington.edu/publications/heacock/, Feb. 17, 1998.

Kaiser Electro-Optics (2752 Loker Avenue West, Carlsbad, Calif. 92008manufactures the "CardioView," "Series 8000," and "StereoSite" HMDdisplay systems for Vista Medical Technologies. These systems are bulky,heavy, and expensive, and include two LCD display devices. Forperipheral vision correction they require the user to wear the HMD overconventional corrective eyeglasses, aggrevating user inconvenience anddiscomfort.

Attempts to use only a single display device for such applications havetypically involved beamsplitters, and have not achieved truestereographic performance (see for example Meyerhofer et al. U.S. Pat.No. 5,619,373, issued Apr. 8, 1997).

Therefore, what is needed in the art is a compact, high resolution, highcontrast, truly stereographic system for microdisplay viewing,particularly for surgical microdisplay viewing, that is suitable forhead-mounted display use without requiring undue complexity or expense.The system should provide good color fidelity for color image viewing,and should incorporate ergonomic design for comfort and efficiency,including peripheral vision accommodation and minimal cabling.

SUMMARY OF THE INVENTION

In accordance with the invention, true stereographic viewing is achievedusing a single display device with appropriate less expensive optics andwithout a beamsplitter. Low-cost complex plastic optics allow biocularviewing of a single electro-optic display device, such as for use in ahead-mounted display (HMD). A dual off-axis configuration provides twoindependent optical channels, intersecting only at the image surface ofthe display device. Each optical channel contains its own illuminationsource, eyepiece lens, and imaging optics. In some embodiments, nearlycollimated illumination optics and intermediate field lenses are used tofill wide-aperture eyepieces without the need for a beamsplitter.

Multiple illumination schemes are provided for either monochrome orcolor, and in either two-dimensional or time-sequential truestereographic presentation. In some embodiments, true stereographicperformance is achieved by sequential activation of the light sources inthe two channels in synchronism with sequential video signals for therespective channels. Offsetting color overcorrection and undercorrectionmethods are applied to minimize optical element complexity. Additionalfeatures include lightweight eyepiece construction and interchangeablelenslets for peripheral vision correction.

Some embodiments include a video interface, which converts conventionalRGB-VGA formatted video signals to sequential color data for storage inintermediate refresh frame memory. Some versions of the video interfaceincorporate wireless transmission, eliminating cumbersome cabling.

Therefore, in some embodiments of the invention, a compact, highresolution, high contrast, truly stereographic system is provided formicrodisplay viewing, that is suitable for head-mounted display usewithout requiring undue complexity or expense. The system provides colorcorrection for good color fidelity, and incorporates ergonomic featuresfor comfort and efficiency, including peripheral vision accommodation.Wireless versions eliminate the need for cumbersome cabling.Particularly, high resolution, high color fidelity, truly stereographicviewing in a head-mounted display, including peripheral visionaccommodation and wireless transmission, provides the depth perceptionand convenience required for surgical remote viewing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of an unfolded optical path off-axis biocularsystem, in accordance with the invention;

FIGS. 2A-2G are schematic views of thin color-corrected lens elements,illustratively for use in a biocular display system, in accordance withthe invention;

FIGS. 3A-3D are cross-sectional views of a folded optical path,reflective display biocular system;

FIGS. 4A and 4B are elevational views of a biocular display viewingsystem folded into a head-mounted display (HMD) housing, in accordancewith the invention;

FIG. 4C is a top view illustrating schematically a biocular displayviewing system installed into a head-mounted display (HMD) housing;

FIG. 4D is an elevational view illustrating a photodetector mounted withan input lens on a HMD housing;

FIG. 5 is a simplified schematic block diagram of a circuitinterconnecting a photodetector with display light sources;

FIG. 6A is a block diagram illustrating the generation, transmission,reception, and processing of a video signal into a format suitable for adisplay system; and

FIG. 6B is a block diagram illustrating the functions of video signalprocessing modules in connection with the use of intermediate framememory with a display system.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The invention relates generally to image display systems and moreparticularly to dual off-axis stereographic systems with multipleillumination sources for biocular viewing of single microdisplays.

In some embodiments of the invention, a compact biocular opticalconfiguration provides a wide-angle image of a single display surfaceindependently to each eye.

FIG. 1 is a schematic view of an unfolded optical path off-axis biocularsystem 100, in accordance with the invention, incorporating a singletransmissive display device 110 having a substantially planar displaysurface 108, as described in detail below. Display device 110 ispositioned at the intersection of two independent beams 112 and 114 ofnearly collimated light, defining respective substantially rectilinearbeam axes 113 and 115. In some embodiments of the invention, independentbeams 112 and 114 are individually produced from a single or multiplelight sources, as described in greater detail below. Independent beams112 and 114 preferably each have collimated beam width (e.g. diameter116) approximately equal to the width (e.g. diameter) of display surface108 projected onto planes perpendicular to respective beam axes 113,115. Beam axes 113 and 115 are tilted relative to one another and to adisplay axis 117 perpendicular to display surface 108.

Independent beams 112 and 114 propagate through display surface 108along respective beam axes 113 and 115 to respective intermediate imageplanes 122 and 124. Single or multiple imaging elements 126 and 128 formintermediate images 132 and 134 of display surface 108 in space atintermediate image planes 122 and 124 respectively. In some embodiments,imaging elements 126 and 128 incorporate toroidal optical correction tocorrect for off-axis geometric distortion.

Intermediate image planes 122 and 124 are reimaged by respectiveindependent eyepiece lenses 136 and 138, configured to magnifyintermediate image planes 122 and 124 respectively and to providevirtual images (not shown) for independent viewing by each eye 146, 148of an observer. Because of the collimated nature of independent beams112 and 114 at display surface 108 and the specific paths of theresultant beams through intermediate image planes 122 and 124, theoptical energy of respective independent beams 112 and 114 does not fillthe eyepiece apertures 142, 144. It has long been recognized by thosehaving ordinary skill in the art, that optical energy must fill eyepieceapertures 142, 144 sufficiently in order to provide complete and uniformvirtual images to the eyes 146, 148 of an observer.

In some embodiments, this condition is met in a traditional manner byinserting a field lens 152, 154 (see for example F. A. Jenkins and H. E.White: "Fundamentals of Optics," McGraw-Hill New York, Toronto, London3rd edition 1957, pp. 182-183) along beam axis 113, 115 at or nearintermediate image plane 122, 124 to fill its respective eyepieceaperture 142, 144 for all possible fields of view. By definition, fieldlenses as described herein optionally include scattering screens andFresnel lenses (see for example The Photonics Dictionary 1988, LaurinPublishing Co., Inc., Pittsfield, Mass.). In some embodiments, fieldlens 152, 154 is given on-axis or off-axis toroidal power to correctresidual image distortions and/or to accommodate the field curvature ofthe eyepiece design.

Independent beam 112 propagating along beam axis 113, together withintermediate imaging plane 122, imaging element 126, eyepiece lens 136,eyepiece aperture 142, and any other optical elements lying alongindependent beam 112 are defined collectively as left optical channel140. Likewise independent beam 114 propagating along beam axis 115,together with intermediate imaging plane 124, imaging element 128,eyepiece lens 138, eyepiece aperture 144, and any other optical elementslying along independent beam 114 are defined collectively as rightoptical channel 150. Left and right optical channels 140 and 150intersect one another only at the display device 110. Off-axis biocularsystem 100 avoids the loss of light, bulk and expense of conventionalbeamsplitters, but illumination off-axis relative to of display axis 117induces geometric distortion of the image of display surface 108, whichmust be accommodated or compensated. Accordingly the tilt angle betweenbeam axes 113 and 115 is kept small enough to minimize geometricdistortion but large enough so that optical components of the respectiveoptical channels 140, 150 do not physically conflict with one another.

In some embodiments, left optical channel 140 uses single or multiplelight-emitting diodes (LEDs) 162 as light sources, and their emissionsare mixed and concentrated within a mixing light cone 166 to produce anapproximate point light source 172. Light cones are familiar in the artfor collecting and concentrating light, and can be fabricated readily bymethods including machining, molding, casting, and electroforming.

Light from point light source 172 is collected and collimated by acollimating lens 176 to produce substantially collimated independentbeam 112. Independent beam 112 is obliquely incident on substantiallyplanar display device 110, e.g. a miniature liquid-crystal display.Imaging element 126 in left optical channel 140 projects intermediateimage 132 of display surface 108 at intermediate image plane 122, wherefield lens 152 directs the optical energy in left optical channel 140 tofill left eyepiece aperture 142 uniformly. Left eyepiece lens 136 formsa virtual image (not shown) of intermediate image plane 122 forcomfortable viewing.

In some embodiments, imaging element 126 incorporates toroidalcorrection to compensate for geometric distortion arising from theoff-axis imaging of display surface 108.

FIG. 1 also shows an aperture stop 182 inserted in at a plane of minimumconvergence 186 of independent beam 112 between imaging element 126 andintermediate image plane 122. As is familiar in the art, aperture stop182 functions primarily to optimize contrast by blocking glare fromstray and scattered light. If independent beam 112 is sufficientlycollimated where it intersects display surface 108, then plane ofminimum convergence 186 becomes an approximate Fourier transform plane(hereinafter the transform plane). Since independent beam 112 isincoherent, the transform plane is a an approximate power Fouriertransform plane and not an amplitude Fourier transform plane, whichexists only in systems having coherent light. The detail in the displaystructure and image diffracts light at a small angle away from beam axis113, and consequently this diffracted light propagates through thetransform plane displaced slightly off-axis relative to nondiffractedlight, which propagates substantially along beam axis 113. As isfamiliar in the art, the off-axis displacement of the diffracted lightat-the transform plane is proportional to the spatial frequency in thescattering display structure or image. Aperture stop 182 isappropriately sized to transmit spatial frequencies corresponding withthe desired image resolution (for example VGA), while blending finerdetail due to the display device's sub-pixel structure by blockinghigher-order spatial frequency harmonics (e.g. light displaced fartheroff-axis at the transform plane), thereby reducing "graininess" andimproving image quality.

Likewise in some embodiments, right optical channel 150 uses single ormultiple light-emitting diodes (LEDs) 164 as light sources, and theiremissions are mixed and concentrated within a mixing light cone 168 toproduce an approximate point light source 174. Light from point lightsource 174 is collected and collimated by a collimating lens 178 toproduce substantially collimated independent beam 114. Independent beam114 is obliquely incident on substantially planar display device 110.Imaging element 128 in right optical channel 150 projects intermediateimage 134 of display surface 108 at intermediate image plane 124, wherea field lens 154 directs the optical energy in right optical channel 150to fill right eyepiece aperture 144 uniformly. Right eyepiece lens 138forms a virtual image (not shown) of intermediate image plane 124 forcomfortable viewing.

In some embodiments, imaging element 128 incorporates toroidalcorrection to compensate for geometric distortion arising from theoff-axis imaging of display surface 108.

In some embodiments, FIG. 1 also shows an aperture stop 184 inserted atthe plane of minimum convergence 188 of independent beam 114 propagatingtoward intermediate image plane 124. As described above in relation toaperture stop 182, aperture stop 184 functions primarily to optimizecontrast by blocking glare from stray and scattered light. Ifindependent beam 114 is sufficiently collimated where it intersectsdisplay surface 108, then plane of minimum convergence 188 is also atransform plane. As described above in relation to aperture stop 182,aperture stop 184 is appropriately sized to transmit spatial frequenciescorresponding with the desired image resolution, while blending finerdetail due to the display device's structure by blocking higher-orderspatial frequency harmonics, thereby reducing "graininess" and thusimproving image quality.

Miniature display devices are available in both monochrome and colorversions and in either transmissive or reflective configurations.Suppliers of transmissive microdisplays include Sony Semiconductor ofSan Jose, Calif.; and the Sharp, Canon, Seiko, Epson, and Kopincompanies of Japan. Suppliers of reflective microdisplays include CRL ofDawley Road, Hayes, Middlesex, UK; CMD of Boulder, CO; Displaytech, Inc.of Longmont, CO; and Varitronix Limited of Hong Kong. A reflectivemicromirror display device is produced by Texas Instruments of Dallas,Tex.

Color display devices are generally supplied with microdot or microstripcolor filters, requiring at least three pixels (red, green, and blue) tocreate a single color display element. As will be apparent to oneskilled in the art, this compromises resolution and sacrifices opticalefficiency. For display devices that are capable of sub-frame rateswitching speeds, another option is to apply to each pixel sequentialcolor techniques such as those used in early television with colorwheels or, more recently, electro-optic switchable filters.

In some embodiments of the invention, light-emitting diodes 162 of leftoptical channel 140 comprise a group of red, green and blue color LEDs,for example 162r, 162g, 162b. LEDs 162r-g-b illuminate the inputaperture to mixing light cone 166. Likewise light-emitting diodes 164 ofright optical channel 150 comprise a group of red, green and blue colorLEDs, for example 164r, 164g, 164b, which illuminate the input apertureto mixing light cone 168. LEDs 162r-g-b, 164r-g-b are sequentiallyswitched for color in any manner that is compatible with anappropriately fast display device 110. For example, red LEDs 162r, 164rare switched on and all other LEDs are switched off synchronously, whilea red frame video signal is applied to display device 110. For long-termstability of LED output powers and associated display color balance,electro-optic detectors 192, 194 are positioned to sample the opticaloutputs within appropriate color bands in respective left and rightoptical channels 140, 150 and appropriately adjust the drive currentswith closed-loop electronic feedback.

In some embodiments the outputs from LEDs 162, 164 are captured bydiffractive collectors 170, 171 and are mixed and homogenized in mixinglight cones 166, 168, as described in detail below. Diffractivecollectors 170, 171 are configured to diffract and superimpose therespective color components from LEDs 162r-g-b within mixing light cones166, 168.

Some biocular vision systems and computer applications require truestereographic display viewing. A well established technique usestime-domain multiplexing of alternating right and left video signalswith synchronous right-eye and left-eye electro-optic shutters (see forexample Meyerhofer et al. U.S. Pat. No. 5,619,373; also Tektronix, Inc.,Beaverton, Oreg., SGS 430 System bulletin). Similar products are offeredby VRex (Elmsford, N.Y.), 3DTV Corporation (for example Model DMMStereoPlate), Kasan Model 3DMax™, PCVR™, and Stereospace Model 1™.

In some embodiments of the invention, true stereographic viewing isprovided by alternate sequencing of the right and left groups of LEDs162, 164, synchronously with time-domain multiplexing of alternate rightand left video signals. Unlike previous systems that combine time-domainmultiplexing of alternating right and left video signals with a commonillumination source for both right-eye and left-eye optical channels(see Meyerhofer et al. U.S. Pat. No. 5,619,373), the present embodimentprovides for alternate sequencing of separate right and left groups ofLEDs 162, 164 for separate optical channels 140, 150 respectively.Off-axis biocular system 100 provides unique true stereographic viewingcapability with a single display device 110, because left and rightoptical channels 140 and 150 remain optically independent of one anotherfrom illumination sources 162, 164 to the observer's eyes 146, 148,except for sharing common display device 110. In some embodimentsstereographic video sequencing is performed concurrently with theabove-described sequential color switching at a sequencing rate and in acombination that minimizes visual flicker.

In some embodiments, eyepiece lenses 136, 138 comprise reflective and/orrefractive lens elements. For a compact optical geometry facilitating awide angular field of view, refractive eyepiece lenses are preferable.In some embodiments, eyepiece lenses 136, 138 are color-corrected(achromatic) to enhance color and image fidelity. Eyepiece lenses 136,138 have eye relief ranging from approximately 0.5 inch to 0.7 inch andangular aperture ranging from approximately 20 degrees to 40 degreesfull angle. In some embodiments eyepiece lenses 136, 138 are furtherrefined by using surfaces with higher order curvatures (asphericsurfaces) to minimize aberrations. Such lenses tend to be quite thickand heavy, generally require multiple elements for achromaticperformance, and are difficult and expensive to fabricate. Conventionalplastic injection-molded lenses can utilize the opposite colordispersions of refractive and diffractive surfaces to create simpler,lighter, and less expensive achromats, but the appropriate lenses aretypically delicate and tend to scatter light.

FIGS. 2A-2G are schematic views of thin color-corrected lens elements,illustratively for use in biocular display system 100, in someembodiments of the invention. FIG. 2A shows a conventional singletrefractive lens 210, having significant color distortion. Illustrativelyred, green, and blue color images of point object 208 are focused insequence at points labeled R, G, and B respectively (exaggerated forclarity). FIG. 2B shows a continuously-profiled Fresnelled refractivesurface 220, having reduced weight and thickness relative toconventional refractive lens 210, but also having color distortion suchthat the red, green, and blue color images of point object 218 arefocused at points labeled R, G, and B respectively, in the same sequenceas for conventional refractive lens 210. FIG. 2C shows a diffractivesurface 230 used as a lens. Diffractive surface 230 is compact andlight-weight, and also has color distortion, but in a reverse sequencerelative to that for lens 210 and Fresnelled refractive surface 220, asillustrated by red, green, and blue color image points of point object228 labeled R, G, and B respectively. As is familiar in the art, twoconventional refractive lenses 210 and 212 having different curvaturesand materials can be combined for color correction, as illustrated bythe superposition of color image points of point object 238 labeled R,G, B in FIG. 2D.

In some embodiments, as is familiar in the art, diffractive structuresare superimposed on Fresnelled refractive surfaces for greater designversatility. Illustratively, FIG. 2E shows diffractive-refractive lens240 combining diffractive surface 230 with Fresnelled refractive surface220. In addition to collapsing the thickness of the lens elements,thereby decreasing their weight and improving their moldability,diffractive-refractive lens 240 applies the offsetting color distortionsequences of diffractive surface 230 and Fresnelled refractive surface220 to achieve color correction, as illustrated by the superposition ofcolor image points of point object 248 labeled R, G, B in FIG. 2E.

In some embodiments, multiple thin optical elements are used for eacheyepiece with any combination of smooth, refractive, and diffractivesurfaces for improved control of optical aberrations. In someembodiments (not shown), the outer surface (eye side) is preferablysmooth for improved cleanability. The more delicate diffractive surfacesare thereby protected as intermediate surfaces. In some embodiments,Fresnelled refractive surfaces 220 and/or diffractive surfaces 230 areapplied to complex substrate curvatures 216 for greater designversatility, as illustrated by a complex refractive-diffractive lens 250in FIG. 2F. Illustratively complex refractive-diffractive lens 250corrects chromatic distortion (superposition of color image pointslabeled R, G. B) for both an on-axis point object 252 and an off-axispoint object 254.

FIG. 2G is a schematic view of a complex refractive-diffractive eyepiecelens 260, incorporating chromatic and geometric correction. Eyepiecelens 260 is placed less than one focal length away from an object 262,and creates a virtual image 264 of object 262 at a distance greater thanone focal length from eyepiece lens 260, as viewed through eyepiece lens260 by an observer 266. Virtual image 264 retains chromatic andgeometric fidelity. Object 262 can be a physical object or a projectedintermediate real image.

As described above, color correction of an eyepiece lens can require asuperposition of multiple refractive and diffractive surfaces applied tocomplex substrate curvatures. This in turn increases manufacturingcosts. In some embodiments, a simpler and less expensive system approachis employed, in which selected optical elements in the system are colorovercorrected, whereas other optical elements are uncorrected orminimally corrected for color. Using such a system approach, overallsystem performance is made achromatic by balancing offsetting colordistortions of individual optical elements.

Illustratively, a diffractive lens surface 230 (see for example FIG. 2C)is applied to imaging elements 126 and 128 (see FIG. 1) either alone orin combination with other optical structures (e.g. toroidal correction).This results in color overcorrection, i.e. red, green, and blue colorimage points labeled R, G, B of point object 228 occur in the sequenceshown in FIG. 2C. If imaging elements 126, 128 are appropriately colorovercorrected, then field lenses 152, 154 and eyepiece lenses 136, 138need not be individually achromatic for overall off-axis biocular system100 to be achromatic. As needed, field lenses 152, 154 and/or eyepiecelenses 136, 138 are minimally color corrected.

It is also advantageous to have chromatic correction at planes ofminimum convergence 186, 188, which are images of point sources 172, 174respectively. To filter the fine sub-image structure requires a smalldiameter aperture stop 182, 184 at planes of minimum convergence 186,188. Chromatic distortion separates the planes of minimum convergencefor different colors, e.g. R, G, B, and thereby causes a loss of imageresolution accompanying the sub-image structure filtering. Chromaticcorrection keeps the planes of minimum convergence 186, 188 in registerfor all colors. Therefore minimal sacrifice of image resolutionaccompanies sub-image structure filtering.

In some embodiments, the optical elements between point sources 172, 174and their images in planes of minimum convergence 186, 188, namelycollimating lenses 176, 178 and imaging elements 126, 128, areindividually non-corrected but are overall corrected chromatically. Inparticular, collimating lenses 176, 178 are color undercorrected andimaging elements 126, 128 are color overcorrected so that the net resultis chromatic correction at planes of minimum convergence 186, 188. Insome embodiments this is combined with color undercorrection of eyepiecelens 136, 138. Since display surface 108 lies in a collimated beam,collimating lens 176, 178 has no effect on image color fidelity ateyepiece lens 136, 138. Therefore the designer is free to use colorovercorrection of imaging elements 126, 128 to compensate for colorundercorrection of eyepiece lens 136, 138, as described above.

Plastic lenses are commonly injection molded from a variety of opticallytransparent materials, including acrylic, polycarbonate, styrene, andnylon. As will be recognized by those skilled in the art, optical designand fabrication methods described in the above embodiments forparticular optical elements are illustrative and are applicable to otheroptical elements in the system, as required for a particularconfiguration. Detailed selection of locations and properties of opticalelements, e.g. focal lengths and aperture diameters, is performed inaccordance with techniques familiar in the art.

FIGS. 3A-3D are cross-sectional views of a folded optical path,reflective display biocular system 300, in accordance with theinvention. This embodiment allows a more compact folded opticalconfiguration, for example for use in a head-mounted display, than doesthe unfolded biocular off-axis system 100 shown in FIG. 1. Except forthe folded optical paths, reflective display biocular system 300performs substantially the same functions as unfolded biocular off-axissystem 100, and each system contains elements that are-essentiallyfunctional counterparts of elements of the other system. Some elementsthat are transmissive in biocular off-axis system 100 are reflective inreflective display biocular system 300. Elements that are substantiallysimilar in the various figures are designated by similar referencenumbers.

FIG. 3A is a top cutaway projection showing a left eyepiece assembly 280and a molded optical assembly 290. A right eyepiece assembly, symmetricwith left eyepiece assembly 280, is not shown for clarity. Left eyepieceassembly comprises a field lens 352 located at or proximate to anintermediate image plane 122, a deflecting mirror 282, and an eyepiecelens 336. In some embodiments eyepiece lens 336 consists of a singlerefractive or refractive-diffractive optical element. In otherembodiments eyepiece lens 336 comprises multiple refractive, Fresnelledrefractive, and/or refractive-diffractive surfaces applied to planar orcurved substrates, as described above in connection with FIGS. 2A-2G.Molded optical assembly 290 and left eyepiece assembly 280 are separatedby half an interocular distance 284 that is adjustable by an interocularadjustment mechanism (not shown), coupled to molded optical assembly 290and left eyepiece assembly 280. Similarly field lens 352 is separatedfrom deflecting mirror 282 by a focus adjustment distance 286,adjustable by a focus adjustment mechanism (not shown) coupled to fieldlens 352 and deflecting mirror 282.

FIG. 3B is a cross-sectional view showing molded optical assembly 290 asviewed across section 3B-3B of FIG. 3A. FIG. 3C is a cross-sectionalview of molded optical assembly 290 showing an illumination path asviewed across section 3C-3C of FIG. 3A, and FIG. 3D is a cross-sectionalview of molded optical assembly 290 showing an imaging path as viewedacross section 3D-3D of FIG. 3A. Molded optical assembly 290 mounts on asubstrate 292 by means of spacers 294. In some embodiments substrate 292is a printed circuit board. Substrate 292 also supports light sources,preferably LEDs 162, 164, and a reflective display device 310 in anominally planar relationship. In some embodiments LEDs 162 comprisered, green, and blue color LEDs 162r-g-b and 164r-g-b respectively (seeFIG. 1).

Molded optical assembly 290 includes total internally reflective (TIR)surfaces 372 and 374, collimating lens 376, imaging elements 326 and328, and TIR stop 382. Molded optical assembly 290 further includesmixing light cones 166, 168, equipped with diffractive collectors 170,171 respectively adjacent LEDs 162, 164.

In some embodiments molded optical assembly 290 is fabricated in wholeor in part as a monolithic, unitary structure with self-containedoptical components, as described in Hebert U.S. Pat. No. 5,596,454,issued Jan. 21, 1997. Integrally molded mechanical features allow forprecise registration of the optical elements of molded optical assembly290 relative to one another and relative to substrate 292 and therebyrelative to LEDs 162, 164 and reflective display device 310 mounted onsubstrate 292. In other embodiments molded optical assembly 290 isfabricated as a frame onto which individual optical elements areassembled.

In operation, a video signal is applied to reflective display device310, thereby producing a reflective video image on reflective displaysurface 308. In some embodiments the video signal comes from a remotevideo camera (not shown). In some embodiments a computer graphicgenerator synthesizes the video signal. In some embodiments the videosignal is supplied from data previously stored on disk, tape, or otherstorage medium. Acquisition and processing of the video signal isdescribed below in greater detail.

Reflective display surface 308 is illuminated by light beams definingindependent left optical channel 340 and right optical channel 350respectively. Arrows labeled 340 in FIGS. 3A-3D represent the beampropagation path of left optical channel 340. The beam propagation pathof right optical channel 350 is symmetric with that of left opticalchannel 340, but is not labeled for clarity. Each optical element ofright optical channel 350 is a symmetric counterpart and performs afunction identical to that of a corresponding optical element of leftoptical channel 340. Beam propagation paths are laid out so that leftoptical channel 340 and right optical channel 350 intersect only atreflective display surface 308, ensuring that optical elements ofrespective left and right optical channels 340 and 350 do not interferephysically with one another. As described above in connection with FIG.1, image distortion is produced in left and right optical channels 340,350 respectively, because of off-axis illumination of reflective displaysurface 308.

Referring to FIG. 3C, illumination for left optical channel 340 isgenerated by LEDs 162. In some embodiments LEDs 162 comprise color LEDs162r-g-b, whereas in other embodiments the illumination is monochrome.The outputs from LEDs 162 are captured by diffractive collector 170 andare mixed and homogenized in mixing light cone 166.

Diffractive collector 170 is configured to deflect and superimpose therespective color components from LEDs 162r-g-b within mixing light cone166. Illustratively each differing color component from LED 162r, 162g,162b respectively impinges on diffractive collector 170 at a differingangle of incidence. Diffractive collector 170 is configured so that therespective color components are diffracted at differing angles, suchthat the diffracted color components are all substantially superimposedand propagate together within mixing light cone 166, as if these colorcomponents were all emitted from a spatially common light source. Thisapplication of diffractive collector 170 is essentially the inverse oftypical diffractive surfaces, wherein a single light beam comprisingdiffering color components is diffracted into a plurality of separatelight beams each having differing angles for differing color components.

Beam 340 emerges from mixing light cone 166 substantially as a pointsource of light, which is deflected by TIR reflective aperture 372.After being collected and reflected by a TIR folding surface 374, beam340 passes through collimating lens 376, producing a substantiallyparallel beam 340 that reflects obliquely from reflective displaysurface 308 (see diagonal arrow 340 in FIG. 3A). At reflective displaysurface 308, beam 340 is spatially modulated by a video image. In someembodiments color LEDs 162r-g-b are sequenced synchronously with a colorvideo signal to reflective display device 310, thereby producing asequentially color modulated reflected beam, as described abovein-connection with FIG. 1.

Referring to FIG. 3D, after modulation and reflection from reflectivedisplay surface 308, beam 340 propagates to imaging elements 326 and328, which collectively perform functions analogous to those of imagingelements 126, 128 as described above in connection with FIG. 1. In thepresent embodiment, imaging element 326 is a transmissive element thatcombines refractive, Fresnelled refractive, and/or diffractiveproperties. Imaging element 328 is a TIR reflective element disposed tofold beam 340 toward TIR stop 382, which then deflects beam 340 towardintermediate image plane 122 (see FIGS. 3A-3B). In some embodiments,imaging element 328 incorporates curvature, which combined withtransmissive imaging element 326, focuses beam 340 through a plane ofminimum convergence at TIR stop 382 and then to form an intermediateimage of reflective display surface 308 at intermediate image plane 122.In some embodiments imaging element 326 and/or 328 incorporates toroidalcorrection to compensate for image distortion arising from off-axisimaging of display surface 308.

TIR stop 382 combines the function of folding beam 340 toward lefteyepiece assembly 280 with an aperture stop function, similar toaperture stops 182, 184 described above in connection with FIG. 1. TIRstop 382 comprises a small TIR element that reflects the central portionof beam 340 at substantially a right angle toward left eyepiece assembly280, while letting undesired diffracted and scattered light passharmlessly around it.

Molded optical assembly 290 advantageously uses TIR surfaces to reflectbeam 340 efficiently without the need for expensive optical coatings.This design approach facilitates unitary fabrication of molded opticalassembly 290.

After reflecting from TIR stop 382, beam 340 propagates to intermediateimage plane 122, where it forms an intermediate image 132 of reflectivedisplay surface 308. The properties of intermediate image 132 of thepresent embodiment are substantially identical with those ofintermediate image 132 described above in connection with FIG. 1. As inthe embodiment of FIG. 1, field lens 352 is placed at intermediate imageplane 122, in order to fill the aperture of eyepiece lens 336 withoptical energy. For compactness a deflecting mirror 282 to fold beam 340is inserted between intermediate image plane 122 and eyepiece lens 336.

The full range of optical element configurations, as described above, isapplicable to reflective display biocular system 300. Theseconfigurations include transmissive and reflective, off-axis, toroidal,refractive, diffractive, and Fresnelled refractive elements. In someembodiments, a system design approach to color correction, as describedabove in connection with FIGS. 2A-2G, is applied to reflective displaybiocular system 300. With such a system design approach, the overallsystem performance is made achromatic by balancing the offsetting colordistortions of individual optical elements.

Illustratively, a diffractive lens surface 230 (see for example FIG. 2C)is applied to imaging element 326 (see FIG. 3D) either alone or incombination with other optical structures (e.g. toroidal correction).This results in color overcorrection, i.e. red, green, and blue colorimage points labeled R, G, B of point object 228 occur in the sequenceshown in FIG. 2C. If imaging element 326 is appropriately colorovercorrected, then field lens 352 and eyepiece lens 336 need not beindividually achromatic for overall reflective display biocular system300 to be achromatic. As needed, field lens 352 and/or eyepiece lens 336is minimally color corrected.

As described above, it is also advantageous to have chromatic correctionat the plane of minimum convergence substantially coincident with TIRstop 382, at which an image of TIR reflective aperture 372 is formed.Chromatic correction keeps the plane of minimum convergence in registerfor all colors, minimizing loss of image resolution with sub-imagestructure filtering.

In some embodiments, the optical elements between TIR reflectiveaperture 372 and its image at TIR stop 382, namely collimating lens 376and imaging elements 326, 328, are individually non-corrected but arecollectively corrected chromatically. For example, collimating lens 376is color undercorrected, but imaging element 326 is color overcorrected,so that the net result is chromatic correction at TIR stop 382. In someembodiments, this is combined with color undercorrection of eyepiecelens 336. Since reflective display surface 308 lies in a collimatedregion of beam 340, collimating lens 376 has no effect on image colorfidelity at eyepiece lens 336. Therefore a system designer is free touse color overcorrection of imaging element 326 to offset colorundercorrection of eyepiece lens 336, as described above.

In some embodiments, true stereographic viewing is provided by alternatesequencing of the right and left groups of LEDs 162, 164, synchronouslywith time-domain multiplexing of alternating right and left videosignals, as described above in connection with FIG. 1. Reflectivedisplay biocular system 300 provides unique true stereographic viewingcapability with a single reflective display device 310, because left andright optical channels 340 and 350 remain optically independent of oneanother, except for sharing common reflective display device 310. Insome embodiments, stereographic video sequencing is performedconcurrently with the above-described sequential color switching at asequencing rate and in a combination that minimizes visual flicker.

FIG. 4A is a front elevational view of a biocular display viewing system400 incorporating a head-mounted display (HMD) housing 410 containing,for example, an off-axis biocular system 100 or a reflective displaybiocular system 300 and incorporating interchangeable correctiveperipheral vision lens elements 422, 424, in accordance with theinvention.

FIG. 4B is a side elevational view of the biocular display viewingsystem of FIG. 4A. For users requiring corrected vision, an appropriateHMD device must either provide eye relief space to accommodateeyeglasses, or provide eyepiece-focusing adjustments. Accommodatingeyeglasses compromises the ergonomic mechanical design of the device inthat the eye relief must be extended beyond the surface of theeyeglasses. Thus, the device becomes bulkier, heavier and generally lesscomfortable.

In some embodiments of the invention, biocular display viewing system400 provides eyepiece-focusing adjustments (not shown). However, trueperipheral vision correction is required for a typical user environment.In some embodiments, interchangeable corrective peripheral vision lenselements 422, 424 are integrated into a HMD housing, as illustrated inFIGS. 4A-4B. As with over-the-counter conventional corrective eyewear, afew general optical prescriptions potentially satisfy a range ofperipheral vision requirements; generally in the range of -0.5 to +1.5diopter. This is especially true for a limited-distance peripheralvision requirement, as in a computer workplace environment.Illustratively, interchangeable corrective peripheral vision lenses 422,424 are made of injection-molded plastic with integrally molded snapfeatures (not shown), which permit easy, interchangeable installationonto a compatibly designed HMD housing 410. Alternatively, commerciallyavailable eyeglass lens blanks may be milled and shaped to fit the HMD.

While many configurations are possible for corrective peripheral visionlens elements 422, 424, a wrap-around configuration as illustrated inFIG. 4A is generally preferred for optimum peripheral vision, whereincorrective peripheral vision lens elements 422, 424 include cutoutsegments to surround respective left and right eyepiece lenses 336, 337.

FIG. 4C is a top view illustrating schematically how a biocular displayviewing system 430 similar to that described in connection with FIGS.3A-3D fits into a head-mounted display (HMD) housing. HMD housing 432(shown in shaded outline in FIG. 4C) with earpieces 434, 436 andcorrective peripheral vision lens elements 422, 424 contains areflective display biocular system 300, incorporating molded opticalassembly 290, as viewed in FIG. 3A. Molded optical assembly 290incorporates a substrate 292 (preferably a printed circuit board) towhich a reflective display device and LED light sources (hidden beneathmolded optical assembly 290 are attached. Left and right imaging beamsemerge from molded optical assembly 290 and are directed to left andright eyes respectively by left field lens 352, left deflecting mirror282, and left eyepiece lens 336, and by right field lens 353, rightdeflecting mirror 283, and right eyepiece lens 337.

Some embodiments include a photodetector with a lens to measure ambientlight intensity and to enable automatic adjustment of the displayintensity for improved visualization. FIG. 4D is an elevational viewillustrating a photodetector 452 mounted with an input lens 454 on HMDhousing 410. Photodetector 452 is disposed to have an ambient field ofview 456 through input lens 454 that approximates the peripheral fieldof view 458 of observer 460. In some embodiments equipped withcorrective peripheral vision lens elements 424 and 422 (see FIG. 4A),peripheral field of view 458 of observer 460 is taken through correctiveperipheral vision lens elements 424 and 422.

FIG. 5 is a simplified schematic block diagram of a circuit 500interconnecting photodetector 452 with display light sources (forexample LEDs 162r-g-b, 164r-g-b, see FIG. 1). Photodetector 452 ispositioned behind input lens 454 and has an output terminal connected toan input terminal of an amplifier 472. Amplifier 472 in turn has anoutput terminal connected to a reference input terminal of a summingjunction 474. A signal input terminal of summing junction 474 isconnected to the output terminal of a nominal contrast-and intensitycontrol module 476. The output terminal of summing junction 474 isconnected to the input terminal of a light source driver 478, which hasan output terminal connected to the light source.

In some embodiments, light source driver 478 represents a plurality oflight source drivers, each driving an individual light source, forexample LED 162r. In some embodiments, circuit 500 represents aplurality of individual circuits comprising individual photodetectors,amplifiers, and light source drivers. In some embodiments, circuit 500includes input terminals (not shown) interconnected with light sourcesequencing apparatus. In some embodiments, all of the elements ofcircuit 500 are contained in or on HMD housing 410. However, otherembodiments apparent to those skilled in the art, having any and all ofthe elements of circuit 500, including photodector 452 and input lens454, located separately from HMD housing 410, are also within the scopeof the invention.

In operation, illustratively input lens 454 collects ambient light overambient field of view 456 and concentrates the collected ambient lighton the sensing element of photodetector 452, which generates an outputsignal proportional to the intensity of the collected ambient light. Theoutput signal from photodetector 452 is amplified by amplifier 472,which generates a reference signal and applies it to the referenceterminal of summing junction 474. Summing junction 474 combines thereference signal with control signals provided by nominal contrast andintensity control module 476, and generates adjusted contrast andintensity control signals, which are applied to the input terminals oflight source drivers 478. The adjusted contrast and intensity controlsignals automatically adjust the intensity of light sources, e.g. LEDs162, 164, for a near-constant ratio to the ambient light intensity,without the need for continual manual intensity adjustment in acontinually changing ambient light environment.

In operation, the display system requires a video input signal. Thevideo signal is typically generated by a source device (e.g. videocamera or VCR) remote from the display.system, and is then transmitted,received, and processed to meet the particular requirements of thedisplay device. FIG. 6A is a block diagram illustrating the generation,transmission, reception, and processing of a video signal into a formatsuitable for a display system, in an embodiment of the invention.

A source device 602 generates video signals in a conventional RGB-VGAformat in the analog domain. These signals are transmitted either bycable (not shown) or by an appropriate wireless data link 606 comprisinga transmitter 608 and a receiving module 610 connected with a processingmodule 612. Processing module 612 is physically located at or adjacentto a display device 620 as described above in connection with FIGS. 1and 3A-3D, or located independently. Optionally a preprocessing module604 (shown in dashed outline) is located at or adjacent to source device602 and is interconnected between source device 602 and transmitter 608.An independently located processing module 612 is typically connected todisplay device 620 by a short cable 614. For example, a processingmodule 612 can be incorporated into an optional head-mounted displayhousing 618 (shown in dashed outline). Alternatively, to reduce bulk andweight of head-mounted display housing 618, processing module 612 isattached conveniently for example to a user's clothing (not shown) andconnected into head-mounted display housing 618 by a short, minimallyencumbering cable 614.

Processing modules 604, 612 convert video signals into a format requiredby display device 620. In practice, visual quality of a display requiresa frame rate that minimizes the eye's perception of flicker. But theflicker frame rate requirement is about four times faster than thatrequired to create a sense of continuous motion of an object within animage.

Unlike conventional analog CRT-based VGA monitors, many miniaturedisplay devices appropriate to the invention are not directlyscan-compatible with the standard RGB interface and require anintermediate memory (frame grabber) for scan format conversion. Whereintermediate memory is a requirement, the display device refreshes fromthe intermediate memory at a frame rate to minimize flicker, but theintermediate memory is updated only at a slower frame rate consistentwith image motion requirements.

FIG. 6B is a block diagram illustrating the functions of processingmodules 604, 612 in connection with the use of intermediate frame memorywith a display system, in an embodiment of the invention. Inputs from aconventional RGB video source (not shown) are received by a serialmultiplexer 640 and a shift register 644 incorporated into processingmodule 604. The vertical sync pulse from the video source causes shiftregister 644 to control multiplexer 640 to generate large sequentialcolor frames of video data. These are combined with vertical andhorizontal sync pulses in a combiner module 642. Optionally, audiosignals are embedded into the horizontal sync pulses of the videowaveform in combiner module 642.

A combined video signal from combiner module 642 is transmitted overwireless date link 606 from transmitter 608 to receiving module 610 andis applied to processor module 612. In processor module 612 the videosignal is optionally amplified in conventional automatic gain controlmodule 630 and is then separated into its respective video, sync, andaudio components by sync separator 632. The various signal componentsare appropriately applied to the intermediate memory (not shown), whichinterfaces with the input electronics of display device 620.

In this case, it is sufficient to serial-multiplex large strings ofanalog video data of each color (frame-by-frame or interleavedline-by-line, for example) prior to transmission to reduce the requiredtransmission data rate, eliminate tracking phase-locked loop clocks, andsimplify the analog-to-digital conversion and memory requirements ofprocessing module 612. This technique results in a slower but visuallyequivalent replication of a standard RGB interface.

In accordance with the invention, true stereographic viewing is achievedusing a single display device with appropriate inexpensive optics andwithout a beamsplitter. Low-cost complex plastic optics allow biocularviewing of a single electro-optic display device, such as for use in ahead-mounted display (HMD). A dual off-axis configuration provides twoindependent optical channels, intersecting only at the image surface ofthe display device. Each optical channel contains its own illuminationsource, eyepiece lens, and imaging optics. In some embodiments, nearlycollimated illumination optics and intermediate field lenses are used tofill wide-aperture eyepieces without the need for a beamsplitter.

Multiple illumination schemes are described for either monochrome orcolor, and in either two-dimensional or time-sequential truestereographic presentation. In some embodiments, true stereographicperformance is achieved by sequential activation of the light sources inthe two channels in synchronism with sequential video signals for therespective channels. Offsetting color overcorrection and undercorrectionmethods are applied to minimize optical element complexity. Additionalfeatures include lightweight achromatic eyepiece construction andinterchangeable lenslets for peripheral vision correction, and automaticcontrast and intensity compensation. A video interface convertsconventionally formatted video signals to serially multiplexed colordata, which fills an intermediate frame refresh memory. Versions of thevideo interface include wireless transmission, eliminated cumbersomecabling.

Particularly, embodiments of the invention provide the depth perception,high resolution, high color fidelity, peripheral vision correction, andconvenience of use in a head-mounted configuration to satisfy thedemanding requirements of medical and surgical remote viewing.

Therefore, in accordance with the invention, a compact, high resolution,high contrast, truly stereographic system is provided for microdisplayviewing, that is suitable for head-mounted display use without requiringundue complexity or expense. The system provides color correction forgood color fidelity, and incorporates ergonomic features for comfort andefficiency, including wide-angle display viewing with long eye relief,and peripheral vision accommodation. Wireless versions eliminate theneed for cumbersome cabling. Particularly, high resolution, high colorfidelity, truly stereographic viewing in a head-mounted display,including peripheral vision accommodation and wireless transmission,provides the depth perception and convenience required for medical andsurgical remote viewing.

Although the invention has been described in terms of a certainpreferred embodiment, other embodiments apparent to those skilled in theart are also within the scope of this invention. Accordingly, the scopeof the invention is intended to be defined only by the claims whichfollow.

I claim:
 1. A method of superimposing and mixing light beams ofdiffering color components, comprising:Collecting light beams ofdiffering color components from a plurality of spatially distinct lightemitting devices on a diffractive collector, said light beams ofdiffering color components impinging on said diffractive collector atdiffering angles of incidence uniquely related to said differing colorcomponents; and diffracting said light beams of differing colorcomponents at said diffractive collector, such that each said light beamof differing color components is diffracted at an angle uniquely relatedto said differing color component, wherein said light beams of differingcolor components are superimposed on one another.
 2. The methodaccording to claim 1, further comprising:directing said superimposedbeams into an input aperture of a mixing light cone; and condensing,mixing, and homogenizing said superimposed beams by reflection insidesaid mixing light cone.